As personal computers (PCs) have become increasingly more powerful, user expectations of the capabilities of such computers have also grown. Less than three decades ago, most computer displays were limited to one or a few colors, and the resolution was relatively low. At that time, most PCs only had a single input device: a keyboard. As simplistic as that might sound by today's standards, early PCs included keyboard buffers to store keystrokes entered by a user in the event the computer could not process the keystrokes quickly enough to keep up with the single-byte-width stream of data being input from the keyboard. Plainly, latent response to user input has long been a concern in computer design.
Certainly, computer design has changed in countless respects. Some of the more popular early PCs used microprocessors running at a clock speed of 4.77 megahertz and employed an 8-bit-external data bus. However, just over two decades later, a typical PC includes a microprocessor running at a clock speed in excess of 3 GHz and features a 128-bit external data bus. In addition, newer and more powerful peripherals have been developed to take advantage of these ever-increasing processing capabilities. As these faster processors have been coupled to image scanners, PC networking systems, and other high-data-volume input/output devices, contention for PC bus cycles has increased. Faster and wider data bus topologies had to be developed to better keep pace with faster microprocessors and the peripheral devices to which the processors are coupled. Over time, the 16-bit, 8 MHZ Industry Standard Architecture (ISA) bus was supplanted by bus structures such as the 32-bit, 8 MHz Extended Industry Standard Architecture (EISA) topology and the 32-bit, near-processor speed Vesa Local Bus structure. Eventually, the Peripheral Component Interconnect (PCI) bus became standard. Today, a typical computer can include a PCI bus featuring a 32-bit or 64-bit data path operating at speeds of 33 MHz or 66 MHz.
As fast as the PCI bus might be, however, it does not alleviate all system throughput bottlenecks. For example, high-resolution video input devices, displays, and graphics applications can require substantial system resources and require relatively large amounts of data to be conveyed for processing by the central processing unit (CPU). Effective graphics applications involve the high-speed transfer of large quantities of data. Latency in acquiring full-motion video data and/or in generating high resolution graphics can reduce the quality of user interaction with the system, hampering the effectiveness and user enjoyment of the system.
Latency in transferring data to a CPU for processing is particularly relevant to new developments in the human-machine interface. These developments are based on the recognition that it is highly desirable to exploit computer graphics to make computers and their interfaces even more user friendly. For example, the MIT Media Lab, as reported by Brygg Ullmer and Hiroshi Ishii in “The metaDESK: Models and Prototypes for Tangible User Interfaces,” Proceedings of UIST 10/1997:14-17,” has developed another form of “keyboardless” human-machine interface. The metaDESK includes a generally planar graphical surface that not only displays computing system text and graphic output, but also receives user input by responding to an object placed against the graphical surface. The combined object responsive and display capability of the graphical surface of the metaDESK is facilitated using infrared (IR) lamps, an IR camera, a video camera, a video projector, and mirrors disposed beneath the surface of the metaDESK. The mirrors reflect the graphical image projected by the projector onto the underside of the graphical display surface to provide images that are visible to a user from above the graphical display surface. The IR camera used in the vision system of the metaDESK can detect IR reflections from the undersurface of an object placed on the graphical surface.
Others have been developing similar keyboardless interfaces. For example, papers published by Jun Rekimoto of the Sony Computer Science Laboratory, Inc., and associates describe a “HoloWall” and a “HoloTable” that display images on a surface and use IR light to detect objects positioned adjacent to the surface.
These new types of human-machine interfaces are intuitive and easy to use. Nonetheless, from a development standpoint, these interfaces are highly complex. To effectively respond to a movement of a physical object across such an interactive surface, data resulting from reflections of IR sources must be rapidly captured and expeditiously presented to the CPU for processing. If a user drags an object across an interactive display surface and the visual response presented to the user lags noticeably behind the user's actions, the interface may be neither pleasing nor workable. Clearly, in some data-intensive applications, small delays due to latent response may be acceptable. For example, in a simple web-cam application where a camera is coupled to a serial input (for example, to a universal serial bus (USB) port), the input data are captured, digitized, buffered while awaiting serial communications to the system, serialized, transmitted, rebuffered, moved to memory, and, ultimately, processed. The resulting delay may be perfectly acceptable for on-line conferencing and similar uses. On the other hand, such performance lags can be completely unacceptable in user interfaces, like the metaDESK and the HoloWall/HoloTable, which rely on computer vision to sense objects on the display surface.
Capture boards and frame grabbers may help to speed computer vision data transfer, but frame grabbers have disadvantages. Frame grabbers with on-board memory may be effective at receiving data from an imaging device, but having sufficient on-board memory to capture a frame of video data can make such devices expensive. Furthermore, the contents of the on-board memory have to be transferred to system memory for processing. Such transfers of large quantities of data, if not managed effectively, burden system throughput and still result in undesired excessive latency.
Ultimately, it would be desirable to be able to receive input data from a sensor imaging apparatus, such as a video camera, and move the data to system memory quickly and efficiently to both avoid latency and without requiring costly hardware devices.